Beyond the Elves: Making Intelligent Agents Intelligent

(Chalupsky et al. 2001, 2002) at the University of Southern California (USC) was to develop software agents to support human organizations. The project was quite successful with impressive prototypes and many papers on the research. In fact, DARPA, which funded the project, was so enthusiastic about the results that it asked us to deploy a version of the Elves for use at DARPA. The original application of the Elves deployed at USC was for the office environment (called the Office Elves) (Scerri, Pynadath, and Tambe 2002; Pynadath and Tambe 2003) and required detailed information about the calendars of people using the system. Deploying this application at DARPA raised a host of issues including privacy and access to internal services. Thus, we decided to deploy a new application of the Electric Elves, called the Travel Elves. This application appeared to be ideal for wider deployment since it could be hosted entirely outside an organization and communication could be performed over wireless devices, such as cellular telephones. The mission of the Travel Elves (Ambite et al. 2002, Knoblock 2004) was to facilitate planning a trip and to ensure that the resulting travel plan would execute smoothly. Initial deployment of the Travel Elves at DARPA went smoothly. Program managers and office directors began using the system. We trained the DARPA travel administrator on how to enter the travel information into the system. But over time things began to go wrong: agents would fail to correctly execute their assigned tasks, information got delayed or was not sent at all, the underlying sources were continually changing, and each user wanted the system to operate in different ways. In this paper, we first review the work on the Travel Elves and describe what the system did and how it worked. Then we describe the real-world problems that arose in deploying this application and the underlying causes of these failures. Finally, we will present some lessons learned and recent research that was motivated by our experiences in deploying the Elves at DARPA.